Military  Operations  Research  Society 


712CD 


73rd  MORSS  CD  Cover  Page 

UNCLASSIFIED  DISCLOSURE  FORM  CD  Presentation 

For  office  use  only  41205 

21-23  June  2005,  at  US  Military  Academy,  West  Point,  NY 

Please  complete  this  form  712CD  as  your  cover  page  to  your  electronic 
briefing  submission  to  the  MORSS  CD.  Do  not  fax  to  the  MORS  office. 


Author  Request  (To  be  completed  by  applicant)  -  The  following  author(s)  request  authority  to  disclose  the 
following  presentation  in  the  MORSS  Final  Report,  for  inclusion  on  the  MORSS  CD  and/or  posting  on  the 
MORS  web  site. 


Name  of  Principal  Author  and  all  other  author(s):  Name:  Robyn  A.  Kane 


Principal  Author’s  Organization  and  address:  Phone:  (719)572-8409 

Fax:  (719)572-8477 

Email: _ rkane@mitre.org 

Original  title  on  712  A/B: _ 

Revised  title: _ AoA  Case  Study:  A  Variation  of  Regret  Analysis 


Presented  in  (input  and  Bold  one):  (WG  27,  CG _ ,  Special  Session _ ,  Poster,  Demo,  or  Tutorial): 


MITRE 


This  presentation  is  believed  to  be: 

UNCLASSIFIED  AND  APPROVED  FOR  PUBLIC  RELEASE  “ 

©  2004  The  MITRE  Corporation.  All  rights  reserved 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

23  JUN  2005 

2.  REPORT  TYPE 

N/A 

3.  DATES  COVERED 

4.  TITLE  AND  SUBTITLE 

AoACase  Study:  A  Variation  of  Regret  Analysis 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

The  MITRE  Corporation  202  Burlington  Road  Bedford,  MA  01730-1420 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

See  also  ADM201946,  Military  Operations  Research  Society  Symposium  (73rd)  Held  in  West  Point,  NY  on 
21-23  June  2005. ,  The  original  document  contains  color  images. 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

uu 

18.  NUMBER 
OF  PAGES 

25 

19a.  NAME  OF 
RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Agenda 

■  Risk  vs  Regret 

■  The  Case  Study  -  ORS  AoA 

-  Capabilities-Based  Approach 

-  ORS  AoA  Process 

■  Cost-Effectiveness  -  A  Deeper  Look 

■  Least  Costly  Launch  Vehicle  Architecture 
Alternatives 

■  Rank  Order  of  Sensitivity  Variability 

■  Average  Regret  Process  &  Results 

■  Minimax  Regret  Process  &  Results 

■  Summary 

■  Study  Impact 

■  Conclusion 


MITRE 


2004  The  MITRE  Corporation.  All  rights  reserved 


Risk  vs  Regret 

■  Risk 

-  Potential  future  harm  that  may  arise  from  some 
present  action 

-Often  combined  or  confused  with  the 
probability  of  an  event  which  is  seen  as 
undesirable 

-Some  theorists  have  defined  quite  general 
methods  to  assess  risk  as  an  expected  after- 
the-fact  level  of  regret 

[http://www.wordiq.eom/definition/Risk#Risk_as_regret] 

■  Regret 

-To  experience  regret  on  account  of;  to  lose  or 
miss  with  a  sense  of  regret;  to  feel  sorrow  or 
dissatisfaction  on  account  of  (the  happening  or 
the  loss  of  something);  as,  to  regret  an  error;  to 
uiTpc  regret  lost  opportunities  orfriends  [Source: 

§j|F§|  |  IvE  ^  11  *  ©2004  The  MITRE  Corporation. 

M/zsMcffiK'c  Doi/icorf  I  /no n/nfinnon/ 


Regret 

■  Regret  associated  with  an  outcome 

-  Defined  as  the  utility  of  that  outcome  subtracted  from  the 
maximum  utility  for  a  given  condition 

-  Measures  the  extent  to  which  one  would  be  unhappy  to 
miss  out  on  an  outcome 

-  Expresses  how  much  worse  things  are  than  they  would 
have  been  had  one  chosen  the  other  option 

■  Calculate  it  by  comparing  the  utility  of  the 
outcome  with  the  maximum  utility  of  that  state 
(condition) 

■  When  the  utility  of  an  outcome  is  the  maximum  for  that 
state  (condition),  the  regret  associated  with  that  outcome 
is  defined  as  zero 


http://www.utas.edu.au/docs/humsoc/philosophy/courses/reasoning/rs/overheads/o17f.htm 
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The  Case  Study  Background 


■  AFSPC/DR  was  commissioned  to  deliver  an  Operationally 
Responsive  Spacelift  (ORS)  Analysis  of  Alternatives  (AoA) 
Study 

■  The  Problem  Statement  was  to  determine  the  best  means  to 
responsively 

-  Launch 

-  Maneuver 

-  Service 

-  Retrieve 

Space  payloads 


To  enhance  military  effectiveness  for  this  purpose 


HOW  TO 
DO  THIS 
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Capabi  lities-Based  Approach 

TRADITIONAL  APPROACH 


ORS  APPROACH 


Alternative  Design 

Cost  Analysis 
Risk  Analysis 
Other  Analysis 


Results  in  Alternatives  with 
Fixed  Utility 


♦  Utility 
■  Cost 
▲  Risk 


Alternatives 
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ORS  AoA  Process:  4  Major  Steps 


1.  Examine  Military 
Utility 

2.  Derive 
responsive 
space  system 
architectures 
and  launch 
loads 

3.  Explore 
responsive 
launch  options 
to  meet  loads 

4.  Determine  most 
cost-  effective 
launch  solution 


Military  Utility  Analysis 


MOP 


MOE=  Measures  of  Effectiveness 
MOP=  Measures  of  Performance 


Results 


LV  Concept  Utility 

Cost 

Risk 

1 

H 

Cl 

R1 

2 

H 

C2 

R2 

3 

H 

C3 

R3 

4 

M 

C4 

R4 

5 

M 

C5 

R5 

6 

M 

C6 

R6 

7 

L 

C7 

R7 

8 

L 

C8 

R8 

9 

L 

C9 

R9 

Payload  Capabilities 
(Surv,  Recon,  Nav) 


Satellite  Architectures 
(7  Identified) 


Launch  Demands 
to  achieve 
L/M/H  Utility 


LV  Options 


LV  Architectures 
(>  85  Identified) 


CONOPS 


Fleet  Size,  Facilities,  Tech.,  Risk,  &  Cost 
to  achieve  L/M/H  Utility 


Cost  Effectiveness  &  Risk  Analysis 
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A  Real  Example  -  ORS  AoA 

■  Effectiveness  Analysis  Working  Group 

Conducted  a  series  of  parametric  experiments  to  establish  three 
levels  of  performance  (low,  medium,  and  high) 

-  Determined  the  military  utility  associated  with  those  levels  of 
performance 

■  Integrated  Concepts  Working  Group 

Used  the  parameters  associated  with  L,  M,  H  performance  levels  as 
the  basis  to  develop  alternatives 

Involved  establishing  eight  Space  Asset  Deployment  Profiles  (SADP) 
as  alternative  means  to  achieve  the  tasks  required  of  the  on-orbit 
assets 

-  Devised  a  series  of  Spacelift  Architecture  Options  to  span  the 
spectrum  of  potential  approaches  to  responsive  launch 

■  Different  launch  capacities  combined  with  these  fundamental 
approaches,  the  set  of  Launch  Vehicle  Architectures  (LVAs)  >  85 

■  Reduced  to  15  LVAs  for  detailed  analysis,  and  costed  for  the 

SADPs _ 

Fixed  performance  across  alternatives  at  a  given  performance  level, 
enables  the  alternatives  to  be  evaluated  strictly  on  cost  and  risk. 


MITRE 
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ORS  AoA  Space  Asset  Deployment  Profiles 


Reference 

Space 

Architecture 
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Cost-Effectiveness:  A  Deeper  Look 


The  Cost  Effectiveness  Working  Group  faced  two  issues: 

■  First  issue:  What  is  the  relationship  between  performance  and 
cost? 

The  LVA  types  were  designed  to  achieve  low,  medium  and  high 
performance 


Cost  Sensitivity  Variability  (Low) 


I  I  ■  I  I 

ELV  #4  RLV  #30  Hybrid  #67  Hybrid  #71  Hypersonic  #64 

LVA  Type 


■  Worst  Case  Delta 

■  Best  Case 


Cost  Sensitivity  Variability  (Medium) 


llll 


I  Worst  Case  Delta 
I  Best  Case 


ELV  #4  RLV  #30 


Hybrid  #67  Hybrid  #71  Hypersonic  #1 

LVA  Type 


Cbst  SerEitivity \&&ilily(Hc^ 


H 


ELVM  FLVfflO  hybidW67  lifaid#71  H/pasjicm 

LVAType 


Best  Case:  the  least  expensive  option  is  Hybrid  LVA  #71 
Worst  Case:  least  expensive  option  is  ELV  or  Hybrid  LVA  #  67 


Best  Case  =  Faster  Processing,  Low  Prod  Cost 
Worst  Case  =  Slower  Processing,  High  Prod  Cost 


MITRE 
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Cost-Effectiveness:  A  Deeper  Look 

■  Second  issue:  What  happens  when  you  consider 
performance,  cost,  and  risk  together? 

-  The  uncertainties  in  processing  time  and  production  cost  are 
unknown  -  only  extremes  were  analyzed 

-  Undertook  a  variation  of  regret  analysis  to  shed  light  on  the 
question 

■  “Regret”  can  be  defined  as  “the  difference  in 
exposure  between  some  choice  and  the  best  choice 
for  a  particular  realization  of  the  uncertainties” 

■  Average  Regret 

-  Is  the  equivalent  of  having  all  equally  likely  outcomes 

■  Minimax  Regret 


Is  equivalent  to  being  pessimistic 
The  “Eeyore”  syndrome 
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Least  Costly  Launch  Vehicle  Architecture 
Alternatives 


■  There  were  15  options  in  the  set  of  narrowed- 
down  LVAs  spanning  the  four  types  of  vehicles 

-  3  Expendables,  8  Reusables,  3  Hybrids,  1  Hypersonic 

■  Included  four  conditions 

-  High  &  Low  Production  Cost 

-  Faster  &  Slower  Processing 

■  Hybrid  #71  has  the  lowest  Life  Cycle  Cost 
Estimate  for  low  performance... but  that  does  not 
tell  the  whole  story 
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ORS  AoA  Launch  Vehicle  Architectures 


(LVAs)* 


LVA  # 

Configuration 

Vehicle  A 

Vehicle  B 

Vehicle  C 

Exp  LVAs 

4 

Sol  25  k 

Exp-Generic  EELV  45  k 

11 

RPRP  25  k 

Exp-Generic  EELV  45  k 

12 

RPRP  45  k 

Reuseable  LVAs 

30 

RPOpt  15  k 

RPOpt  Growth  to  Trimese 

36 

RPLHOpt  5  k 

RPLHOpt  Growth  to  45  k 

39 

RPLHOpt  25  k 

Exp-Generic  EELV  45  k 

41 

RPLHOpt  45  k 

46 

LH  Bimese  15k 

LHBi  Growth  to  45  k 

47 

LH  Bimese  15k 

LHBi  Growth  to  Trimese 

48 

LH  Bimese  25  k 

Exp-Generic  EELV  45  k 

51 

LH  Bimese  45  k 

Hypersonic  LVAs 

64 

RBCC  25  k 

Exp-Generic  EELV  45  k 

Hybrid  LVAs 

67 

RLV/ELV-Solid  RD180 

Exp-Generic  EELV  25  k 

Exp-Generic  EELV  45  k 

70 

RLV/ELV  12.7k 

Hybrid  2  RLVs/ELV  45k 

71 

RLV/ELV  19.5k 

Hybrid  Growth  to  Trimese 

*This  presentation  will  not  discuss  the  details  of  the  LVs 
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Rank  Order  of  Sensitivity  Variability 


Space  Architecture 
Performance 
Processing  Time 
Production  Cost 

Arch  2  Arch  2  Arch  2 

Low  Low  Low 

Faster  Faster  Slower 

High  Low  High 

Arch  2 
Low 
Slower 
Low 

Arch  2 
Medium 
Faster 
High 

Arch  2 
Medium 
Faster 
Low 

Arch  2 
Medium 
Slower 
High 

Arch  2 
Medium 
Slower 
Low 

Arch  2 
High 
Faster 
High 

Arch  2  Arch  2  Arch  2 

High  High  High 

Faster  Slower  Slower 

Low  High  Low 

Expendable  (4) 

6 

12 

2 

9 

4 

11 

1 

4 

4 

10 

1 

2 

Expendable  (11) 

8 

13 

3 

10 

5 

13 

2 

5 

5 

12 

3 

6 

Expendable  (12) 

13 

15 

6 

12 

8 

14 

4 

6 

6 

13 

2 

5 

Reusable  (30) 

5 

2 

11 

8 

6 

4 

8 

8 

7 

5 

10 

11 

Reusable  (36) 

11 

10 

10 

11 

12 

10 

9 

9 

11 

8 

7 

7 

Reusable  (39) 

4 

7 

7 

5 

9 

9 

11 

12 

12 

11 

12 

12 

Reusable  (41) 

7 

3 

8 

4 

7 

5 

7 

7 

8 

6 

9 

9 

Reusable  (46) 

12 

8 

14 

13 

10 

7 

10 

10 

10 

7 

11 

10 

Reusable  (47) 

14 

9 

15 

14 

11 

6 

12 

11 

9 

4 

8 

8 

Reusable  (48) 

10 

11 

9 

7 

14 

12 

13 

14 

14 

14 

13 

14 

Reusable  (51) 

9 

4 

12 

6 

13 

8 

14 

13 

13 

9 

14 

13 

Hypersonic  (64) 

15 

14 

13 

15 

15 

15 

15 

15 

15 

15 

15 

15 

Hybrid  (67) 

2 

6 

1 

2 

2 

3 

3 

2 

1 

3 

4 

1 

Hybrid  (70) 

Hybrid  (71) 

C  1 

C  f 

5 

4 

d 

3  2  5 

Cb  (To_ e 

a L 

cb 

5 

6 

3 

4 

What’s  going  on  between  #71  &  #67? 

For  Low  production  cost,  5  of  6  times  #71  wins  (Red  oval) 

For  High  production  cost,  2  of  6  times  #71  wins  (Black  oval) 

For  Faster  processing  time,  #71  ranks  first  5  of  6  times. 

For  Slower  processing  time,  #71  ranks  first  2  of  6  times. 

Are  Low  production  cost  and  Faster  processing  time  really  possible? 
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Process  to  Calculate  Average  Regret 

1.  Compare  Cost  of  each  LVA  to  Hybrid  LVA  #71  to 
identify  a  delta 

-  If  delta  is  positive,  LVA  #71  cheaper 

-  If  delta  cost  is  negative,  LVA  #71  is  more  expensive 

2.  Calculate  the  four  deltas  at  each  performance  level 
for  the  four  conditions: 

■  Low  Prod  Cost,  Faster  Processing  Time 

■  High  Prod  Cost,  Faster  Processing  Time 

■  Low  Prod  Cost,  Slower  Processing  Time 

■  High  Prod  Cost,  Slower  Processing  Time 

3.  Sum  delta  costs  (step  2)  across  the  four  conditions 

4.  Calculate  average  regret  value  (step  3  divided  by  4) 

-  If  average  delta  is  positive,  LVA  #71  is  cheaper 

■  Issues/Concerns 

-  Each  condition  does  not  have  the  same  probability 


MITRE 
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ORS  AoA  Average  Regret 

(All  vs  Hybrid  LVA  #71 )  _ 


Architecture 


■  Positive  regret  values 
reflect  average  potential 
savings  (BY03$M)  by 
choosing  Hybrid  LVA 
#71  concept  over  other 
LVA  concepts 

■  Negative  regret  value 
reflect  average  potential 
costs  by  choosing 
Hybrid  LVA  #71  concept 
over  that  LVA  concept 


SA2 


SA  2 


SA  2 


Performance 


Low 


Medium 


High 


Average  Regret  versus  LVA  #71 

(+  is  good  for  LVA  #71) 

Expendable  (4) 

15,596 

11,727 

-6,034 

Expendable  (11) 

19,592 

18,455 

4,764 

Expendable  (12) 

29,776 

27,023 

4,299 

Reusable  (30) 

20,235 

50,253 

58,743 

Reusable  (36) 

28,342 

62,334 

54,426 

Reusable  (39) 

14,241 

66,073 

89,367 

Reusable  (41) 

16,110 

47,311 

54,096 

Reusable  (46) 

40,830 

62,298 

62,382 

Reusable  (47) 

45,611 

64,472 

52,762 

Reusable  (48) 

24,554 

87,401 

108,274 

Reusable  (51) 

23,207 

86,647 

98,980 

Hypersonic  (64) 

48,274 

153,307 

228,433 

Hybrid  (67) 

2,174 

-496 

-10,688 

Hybrid  (70) 

8,150 

5,974 

-1,435 
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Minimax  Regret  Analysis 


■  Decisions  made  with  minimizing  the  maximum  regret 
are  to  control  the  remorse  that  inevitably 
accompanies  hindsight 

■  Minimax  Regret  is  distribution  independent  which 
gives  it  its  power  (not  uniformly  distributed) 

■  All  other  LVAs  were  compared  to  the  least  costly 
LVA  in  each  case  within  each  performance  level. 

This  regret  function  is: 

Minimax  Regret  (of  a  particular  alternative  it)  =  (LCCE  of 
other  LVA  #)  -  (Least  costly  LCCE) 
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ORS  AoA  Minimax  Regret  Process  &  Results 

1.  Find  lowest  LCCE  for  each 
condition 

2.  Calculate  regret  value  by 
subtracting  the  lowest  LCCE 
(step  1)  from  each  of  the  other 
LVAs  within  the  respective 
condition 

3.  Obtain  maximum  regret  value 
for  each  LVA  by  taking  the 
largest  regret  value  of  the 
four  conditions 

4.  Choose  the  minimum  of  the 
maximum  regret  values  for 
each  performance  level 

5.  Examine  the  first  and  seccmd 
choices  for  inconsistencies 

(Due  to  complexity  of  this  process,  steps  will 

be  shown  in  detail) 


Space  Arch 

2 

Low 

Med 

High 

Max  Regret  (BY03$M) 

Expendable  (4) 

33,563 

39,004 

35,574 

Expendable  (11) 

39,881 

47,482 

47,305 

Expendable  (12) 

49,943 

56,713 

48,609 

Reusable  (30) 

51,055 

130,510 

185,371 

Reusable  (36) 

50,236 

133,708 

155,299 

Reusable  (39) 

29,269 

146,831 

233,414 

Reusable  (41) 

36,083 

116,009 

172,178 

Reusable  (46) 

86,072 

145,051 

187,384 

Reusable  (47) 

95,956 

150,581 

171,974 

Reusable  (48) 

38,535 

178,474 

257,857 

Reusable  (51) 

51,900 

200,298 

267,315 

Hypersonic  (64) 

66,475 

281,238 

478,988 

Hybrid  (67) 

14,138,, 

(Jvio J 

Hybrid  (70) 

_^666j 

*  47,700 

Hybrid  (71) 

26,226 

64,576 

^CPMax^egret 

Hybrid 

Hybrid 

Hybrid 

(71) 

(67) 

(67) 

13,748 

14,777 

21,107 

Delta  between  71  &  67: 

-390 

1 1 ,449 

43,468 

MITRE 
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ORS  AoA  Minimax  Regret  Process  &  Results 


1.  Find  lowest  LCCE  for 
each  condition 

2.  Calculate  regret  value  by 
subtracting  the  lowest 
LCCE  (step  1)  from  each 
of  the  other  LVAs  within 
the  respective  condition 

3.  Obtain  maximum  regret 
value  for  each  LVA  by 
taking  the  largest  regret 
value  of  the  four 
conditions 

4.  Choose  the  minimum  of 
the  maximum  regret 
values  for  each 
performance  level 

5.  Examine  the  first  and 
second  choices  for 
inconsistencies 


Architecture 

SA2 

SA2 

SA2 

SA2 

Performance 

Low 

Low 

Low 

Low 

Processing  time 

Faster 

Faster 

Slower 

Slower 

Production  Cost 

High 

Low 

High 

Low 

Expendable  (4) 

20,679 

33,563 

5,141 

16,747 

Expendable  (11) 

26,996 

39,881 

6,817 

18,423 

Expendable  (12) 

37,058 

49,943 

17,122 

28,728 

Reusable  (30) 

17,876 

9,278 

51,055 

16,479 

Reusable  (36) 

34,538 

22,463 

50,236 

19,879 

Reusable  (39) 

16,488 

14,815 

29,269 

10,140 

Reusable  (41) 

21,165 

12,022 

36,083 

8,920 

Reusable  (46) 

36,394 

19,843 

86,072 

34,759 

Reusable  (47) 

39,855 

20,681 

95,956 

39,699 

Reusable  (48) 

33,320 

24,394 

38,535 

15,716 

Reusable  (51) 

27,823 

12,678 

51,900 

14,174 

Hypersonic  (64) 

55,760 

44,796 

66,475 

39,813 

Hybrid  (67) 

5,352 

14,138 

0 

2,952 

Hybrid  (70) 

10,183 

13,502 

14,666 

7,996 

Hybrid  (71) 

0 

0 

13,748 

0 
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ORS  AoA  Minimax  Regret  Process  &  Results 


1.  Find  lowest  LCCE  for  each 
condition 

2.  Calculate  regret  value  by 
subtracting  the  lowest 
LCCE  (step  1)  from  each  of 
the  other  LVAs  within  the 
respective  condition 

3.  Obtain  maximum  rearet 
value  for  each  LVA  by 
taking  the  largest  regret 
value  of  the  four  conditions 

4.  Choose  the  minimum  of  the 
maximum  regret  values  for 
each  performance  level 

5.  Examine  the  first  and 
second  choices  for 
inconsistencies 


Architecture 

SA  2 

Performance 

Low 

Expendable  (4) 

33,563 

Expendable  (11) 

39,881 

Expendable  (12) 

49,943 

Reusable  (30) 

51,055 

Reusable  (36) 

50,236 

Reusable  (39) 

29,269 

Reusable  (41) 

36,083 

Reusable  (46) 

86,072 

Reusable  (47) 

95,956 

Reusable  (48) 

38,535 

Reusable  (51) 

51,900 

Hypersonic  (64) 

66,475 

Hybrid  (67) 

14,138 

Hybrid  (70) 

14,666 

Hybrid  (71) 

13,748 
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ORS  AoA  Minimax  Regret  Process  &  Results 


i 


Find  lowest  LCCE  for  each 
condition 

Calculate  regret  value  by 
subtracting  the  lowest 
LCCE  (step  1)  from  each  of 
the  other  LVAs  within  the 
respective  condition 

Obtain  maximum  regret 
value  for  each  LVA  by 
taking  the  largest  regret 
value  of  the  four  conditions 

Choose  the  minimum  of  the 
maximum  regret  values  for 
each  performance  level 

Examine  the  first  and 
second  choices  for 
inconsistencies 


Space  Arch 

2 

Low 

Med 

High 

Max  Regret  (BY03$M) 

Expendable  (4) 

33,563 

39,004 

35,574 

Expendable  (11) 

39,881 

47,482 

47,305 

Expendable  (12) 

49,943 

56,713 

48,609 

Reusable  (30) 

51,055 

130,510 

185,371 

Reusable  (36) 

50,236 

133,708 

155,299 

Reusable  (39) 

29,269 

146,831 

233,414 

Reusable  (41) 

36,083 

116,009 

172,178 

Reusable  (46) 

86,072 

145,051 

187,384 

Reusable  (47) 

95,956 

150,581 

171,974 

Reusable  (48) 

38,535 

178,474 

257,857 

Reusable  (51) 

51,900 

200,298 

267,315 

Hypersonic  (64) 

66,475 

281,238 

478,988 

Hybrid  (67) 

Hybrid  (70) 

14,138 

.  ryvU 

"•K3  748 

►C[4’775 

23,869 
26  226 

HC21J07 

47,700 

64  576 

Mini-Max  Regret 

Hybrid 

Hybrid 

Hybrid 

(71) 

(67) 

(67) 

13,748 

14,777 

21,107 

Delta  between  71  &  67: 

C  -390 

1 1 ,449 

43,468 

_ 
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Comparison  of  Average  vs  Minimax  Regret 

■  Average  Regret 

-  Hides  results  for  the  conditions  where  hybrid  does  not  win 

-  Do  not  have  insight  into  the  combination  of  conditions  (cost 
and  processing  time)  for  which  LVA#71  is  better  than  LVA 
#67 

-  Do  not  know  which  conditions  directly  affected  the  LCCEs  of 
LVA  #67  and  LVA  #71 

■  Minimax  Regret 

-  Revealed  the  hybrid  type  launch  vehicle  had  the  least  regret 
for  all  performance  levels 

-  Through  minimax  regret  analysis,  the  decision-maker  can 
see  which  hybrid  launch  vehicle  to  pursue  if  growth  in 
performance  (low,  medium,  or  high),  funding,  or  risk  were 
priorities 


Average  Regret  &  Minimax  Regret  can  be  powerful  analytical  tools  if 
limitations  of  each  are  known  and  understood 
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Summary  (cont.) 


■  Clearly  there  is  an  interaction  between  the  performance  levels  and 
which  hybrid  is  best 

■  Regret  analysis  verifies  that  hybrid  LVA  does  beat  the  other  LVA 
types  -  even  when  we  assume  the  worst  condition 

■  Regret  analysis  gave  AFSPC  and  other  decision-makers  repeatable, 
traceable  analysis  to  focus  on  the  most  viable  architecture  type 

■  ORS  AoA  recommended  to  pursue  first  stage  reusable  launch 
vehicle  (RLV)  demonstrations 

-  Assess  cost,  operability,  technology  readiness  levels 

-  Test  operations  concepts/processes 

-  Follow-on  demonstrations  of  Hybrid  (reusable  first  stage  with 
expendable  upper  stages) 

Assess  scalability  of  cost  and  operations  data;  test  more  realistic 
operations  concepts 


Demonstrations  will  reduce  cost  uncertainty  as  well  as  technical  risk 
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Study  Impact 

■  Since  completion  of  this  study,  the  HQ 
AFSPC/CC  gave  permission  in  January  2005 
for  the  ORS  Demonstrator  to  begin 

■  A  Phase  I  Program  Research  and 
Development  Announcement  (PRDA)  notice 
was  posted  on  February  16,  2005  for  the 
Affordable  REsponsive  Spacelift-Subscale 
Demonstration  (ARES-SD) 

■  The  ARES  Phase  I  Concept  Development  and 
Demonstration  Planning  Program  is 
contracted  by  the  Department  of  the  Air 
Force,  Air  Force  Space  Command,  SMC  - 
Space  and  Missiles  System  Center 
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Conclusion 


■  Bottom  line:  different  tools  and  types  of 
analysis  yield  different  analysis  results 

■To  achieve  unbiased  analysis,  the  analyst 
cannot  overlook  one  tool  or  type  of  analysis 
over  another 

■When  used  appropriately,  Average  Regret 
and  Minimax  Regret  can  be  powerful 
analytical  tools  if  limitations  of  each  are 
known  and  understood 


25 


MITRE 


2004  The  MITRE  Corporation.  All  rights  reserved 


